Restoring Speech Following Total Removal of the Larynx

By speech articulator movement and training a transformation to audio we can restore the power of speech to someone who has lost their larynx. We sense changes in magnetic field caused by movements of small magnets attached to the lips and tongue. The sensor transformation uses recurrent neural networks

ChunkitApp : Investigating the relevant units of online speech processing

Peer reviewe

Direct Speech Reconstruction From Articulatory Sensor Data by Machine Learning

This paper describes a technique that generates speech acoustics from articulator movements. Our motivation is to help people who can no longer speak following laryngectomy, a procedure that is carried out tens of thousands of times per year in the Western world. Our method for sensing articulator movement, permanent magnetic articulography, relies on small, unobtrusive magnets attached to the lips and tongue. Changes in magnetic field caused by magnet movements are sensed and form the input to a process that is trained to estimate speech acoustics. In the experiments reported here this “Direct Synthesis” technique is developed for normal speakers, with glued-on magnets, allowing us to train with parallel sensor and acoustic data. We describe three machine learning techniques for this task, based on Gaussian mixture models, deep neural networks, and recurrent neural networks (RNNs). We evaluate our techniques with objective acoustic distortion measures and subjective listening tests over spoken sentences read from novels (the CMU Arctic corpus). Our results show that the best performing technique is a bidirectional RNN (BiRNN), which employs both past and future contexts to predict the acoustics from the sensor data. BiRNNs are not suitable for synthesis in real time but fixed-lag RNNs give similar results and, because they only look a little way into the future, overcome this problem. Listening tests show that the speech produced by this method has a natural quality that preserves the identity of the speaker. Furthermore, we obtain up to 92% intelligibility on the challenging CMU Arctic material. To our knowledge, these are the best results obtained for a silent-speech system without a restricted vocabulary and with an unobtrusive device that delivers audio in close to real time. This work promises to lead to a technology that truly will give people whose larynx has been removed their voices back

Voice restoration after laryngectomy based on magnetic sensing of articulator movement and statistical articulation-to-speech conversion

© Springer International Publishing AG 2017.In this work, we present a silent speech system that is able to generate audible speech from captured movement of speech articulators. Our goal is to help laryngectomy patients, i.e. patients who have lost the ability to speak following surgical removal of the larynx most frequently due to cancer, to recover their voice. In our system, we use a magnetic sensing technique known as Permanent Magnet Articulography (PMA) to capture the movement of the lips and tongue by attaching small magnets to the articulators and monitoring the magnetic field changes with sensors close to the mouth. The captured sensor data is then transformed into a sequence of speech parameter vectors from which a time-domain speech signal is finally synthesised. The key component of our system is a parametric transformation which represents the PMA-tospeech mapping. Here, this transformation takes the form of a statistical model (a mixture of factor analysers, more specifically) whose parameters are learned from simultaneous recordings of PMA and speech signals acquired before laryngectomy. To evaluate the performance of our system on voice reconstruction, we recorded two PMA-and-speech databases with different phonetic complexity for several non-impaired subjects. Results show that our system is able to synthesise speech that sounds as the original voice of the subject and also is intelligible. However, more work still need to be done to achieve a consistent synthesis for phonetically-rich vocabularies

A silent speech system based on permanent magnet articulography and direct synthesis

In this paper we present a silent speech interface (SSI) system aimed at restoring speech communication for individuals who have lost their voice due to laryngectomy or diseases affecting the vocal folds. In the proposed system, articulatory data captured from the lips and tongue using permanent magnet articulography (PMA) are converted into audible speech using a speaker-dependent transformation learned from simultaneous recordings of PMA and audio signals acquired before laryngectomy. The transformation is represented using a mixture of factor analysers, which is a generative model that allows us to efficiently model non-linear behaviour and perform dimensionality reduction at the same time. The learned transformation is then deployed during normal usage of the SSI to restore the acoustic speech signal associated with the captured PMA data. The proposed system is evaluated using objective quality measures and listening tests on two databases containing PMA and audio recordings for normal speakers. Results show that it is possible to reconstruct speech from articulator movements captured by an unobtrusive technique without an intermediate recognition step. The SSI is capable of producing speech of sufficient intelligibility and naturalness that the speaker is clearly identifiable, but problems remain in scaling up the process to function consistently for phonetically rich vocabularies

Towards an intraoral-based silent speech restoration system for post-laryngectomy voice replacement

© Springer International Publishing AG 2017,Silent Speech Interfaces (SSIs) are alternative assistive speech technologies that are capable of restoring speech communication for those individuals who have lost their voice due to laryngectomy or diseases affecting the vocal cords. However, many of these SSIs are still deemed as impractical due to a high degree of intrusiveness and discomfort, hence limiting their transition to outside of the laboratory environment. We aim to address the hardware challenges faced in developing a practical SSI for post-laryngectomy speech rehabilitation. A new Permanent Magnet Articulography (PMA) system is presented which fits within the palatal cavity of the user’s mouth, giving unobtrusive appearance and high portability. The prototype is comprised of a miniaturized circuit constructed using commercial off-the-shelf (COTS) components and is implemented in the form of a dental retainer, which is mounted under roof of the user’s mouth and firmly clasps onto the upper teeth. Preliminary evaluation via speech recognition experiments demonstrates that the intraoral prototype achieves reasonable word recognition accuracy and is comparable to the external PMA version. Moreover, the intraoral design is expected to improve on its stability and robustness, with a much improved appearance since it can be completely hidden inside the user’s mouth

Constructing Hierarchical Spheres from Large Ultrathin Anatase TiO2 Nanosheets with Nearly 100% Exposed (001) Facets for Fast Reversible Lithium Storage

Synthesis of nanocrystals with exposed high-energy facets is a well-known challenge in many fields of science and technology. The higher reactivity of these facets simultaneously makes them desirable catalysts for sluggish chemical reactions and leads to their small populations in an equilibrated crystal. Using anatase TiO2 as an example, we demonstrate a facile approach for creating high surface area, stable nanosheets comprised of nearly 100% exposed (001) facets. Our approach relies on spontaneous assembly of the nanosheets into three-dimensional, hierarchical spheres that stabilizes them from collapse. We show that the high surface density of exposed TiO2 (001) facets leads to fast lithium insertion/deinsertion processes in batteries that mimic features seen in high power electrochemical capacitors.X.W.L. and L.A.A. acknowledge support from Award KUS-C1-018-02 made by King Abdullah University of Science and Technology (KAUST). The authors are grateful to the reviewers and Prof. Hua Chun Zeng (National University of Singapore) for very valuable comments and to Nanyang Technological University for financial support through a startup grant (SUG)

Tacit Learning for Emergence of Task-Related Behaviour through Signal Accumulation

International audienceControl of robotic joints movements requires generation of appropriate torque and force patterns, coordinating the kinematically and dynamically complex multijoints systems. Control theory coupled with inverse and forward internal models are commonly used to map a desired endpoint trajectory into suitable force patterns. In this paper, we propose the use of tacit learning to successfully achieve similar tasks without using any kinematic model of the robotic system to be controlled. Our objective is to design a new control strategy that can achieve levels of adaptability similar to those observed in living organism and be plausible from a neural control view point. If the neural mechanisms used for mapping goal expressed in the task-space into control-space related commands without using internal models remain largely unknown, many neural systems rely on data accumulation. The presented controller does not use any internal model and incorporates knowledge expressed in the task space using only accumulation of data. Tested on a simulated two links robot system, the controller showed flexibility by developing and updating its parameters through learning. This controller reduces the gap between reflexive motion based on simple accumulation of data and execution of voluntary planned actions in a simple manner that does not require complex analysis of the dynamics of the system

Creating Pan-Karen Identity: The Wrist Tying Ceremony in the United States

A movement to unite all Karen-language speaking people under the banner of a distinct and unified Karen identity has been afoot for more than a century. Buddhist and Christian Karens from Burma, now living in Fort Wayne, Indiana, USA, are forging – and contesting – a universal Karen identity through the celebration of the “traditional” wrist tying ceremony. This articulation of pan-Karen identity is an example of Baumann and Gingrich\u27s (2004) theory of identity construction, which they call a “grammar of encompassment.” This case study shows that the theory, which claims that instances of encompassment are often contested by the encompassed (that is, the subordinate people whose alterity is denied), must be extended. In the United States, it is the encompassers (that is, the Christians who have greater economic and political power) who are the most deeply troubled by the denial of difference demanded by the discourse of encompassment